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This report gives the basic definition and purpose of 
comp^^tencybased teacher education (CBTE) cut-off scores. It 
describes the basic characteristics of CBTE aa a yes-rio dichotomous 
decilsion regarding the presence of a specific ability or knowledger ^ 
which necesitates the establishment of a cut-off point to designate 
competency -Vs. incompetency on stated objectives. Statistical * 
considerations for establishing CBTE cut-off scores are reviewed, 
and, ba^ed'on test scores, two types of classification errors are 
identified. These are false acceptance, t.e^ir nonmasters erroneously ^ 
classified as masters; and fa-lse rejection, i.e./ masters erroneously • 
classified as nonmasters^ The report recommends that cut-^off scores . 
should not be arbitrarily e"stablished, but should* be based on 
decisi^on theory, the goal of which is to increiase cor^ecft ra^tVand^^- 
decrease error rate. It was*" found that increasing th4 cut-off score 
will deci;rease false acceptance error and increase false rejection 
error, while decreasing the cut-off score will decrease fals^ 
rejection error and increase false accept,ance error. It was concldded 
that since shifting cut-off poin-fcs reduces one error at the expeni^e 
of the other, the direction of ^ift should be in the direction of 
the less serioils error (false rejection). Numerous figures and tables 
are included. (BDj • ' , / 
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STATISTICAL CONSIDERATIONS FOR 
ESTABLISHING CBTE CUT-OPP. SCORES 

^ Joseph A. Trzasko, Ph.D. 

Dept. of Psychology- 
Mercy College 

The "basic definition and purpose of cut-off scores 
is discussed with regard to comj^etenxsy-based teacher 
education (OBTB). Statistical considerations for 
establishing CBTE cut-off scoresj arfe reviewed, , in- 
cluding the concepts of correct /rejection, false • 
acceptance, false rejection, an^L correct accept- 
ancie. The h^isic application of decision theory to 
CBTE is offered. 

A basic characteris-y-c of competency-based teacher 

education (CBTE) is its yes-no dichotomous ^decision regarding 

the presence of a specific ability or knowledge, i.e., does 

a student display competency or incompetency regarding ^ 

required performance or academic material. Therefore, 

^cut-off point must be established to designate competency 

vs. incompetency on stated objectives. Beggs & Lewis (1975, 

pg. 61-62) summarize the ufee of cut-off scores as follows: 

Cut-off scores . One term^ most frequently linked with 
criterion-referenced measurement is' cut-off score . 
The connectlpn between them occtirs because they are 
both used in situations in which the concern is to 
determine whether a student possesses dertain behavior. 
A CTit-off ^cQre is generally applied when a teacher 
is teaching for mastery, attempting to cause students 
to reach a point at which they can answer some pei*- 
^ centage of items on a test correctly. The cut-off 

score is the score the student mu^t obtain before the 
teacher is willing to accept that^ the student, has 
mastered the topic or content under consideration. 
JThe cut-off score is further interpreted to indicate 
the minimal level of the skill ^ being evaluated that ^ 
the student must possess to be successful at the next 
level. Cut-off scores are therefore generally estab- 
lished at the upper end of a, scale, That is, a 
student may be required to respond correctly to 85, 
90, or 95 percent of the items. Qut-off scores and 
criterion-referenced measurement are frequently con- 
fused . because it is popular among teachers, evalua- 
tors, and others to call the cut-off score the 
•^criterion score." ^ . 

Nothing in criterion-referenced mSeasurement requires 
the use of a cut-off score. Criterion-referenced 
measurement is designed to determine whether an indi- 
vidual possesses certain skills. The criterion-vin 



criterion-referenced imeasureiQent is the test item. A 
correct response indicates that the individual possesses 
that skill. An incorrect response indicates that the 
skill has not been achieved. Cut-off scores, on the 
other hand, indicate that the student must achieve some 
^ minimal percentage on a test. The score that the 

student obtains .is interpreted to, mean that the student 
has mastered .that percentage, of the content. 

Cut-off scores should not be arbitrarily established, 

but should b^ based on decision theory. The end-product of 

the correct use of decision theory will be a reduction in 

the number of erroneous judgments. Por example, assume^ that 

a given test is administered to two groups: one group known 

to have mastered the material and another group known not to 

hiave mastered the material. And, as a result of within- 

^roup variability, the, two frequency distributions overlap. 

This is illustrated in Figure 1. The cut-^off point is 



Insert -Figure 1 



set at 70; therefore, accbrding to the test, students 
.scoring 7Q and above are considered malsters, while those 
students scoring below 70 are considered *non-masters. It 
will be noted that based on test scores, two types of classi- 
fication errors will have'' been made: (1) false acceptance, 
/i.e#, non-masteiPs e]^)neously classified as masters^ and, (/2) 
false rejeotioij, i.e., masters erroneously classified as non- 
masters. The overall number of correct decisions or • correct 
rate' is the frequency of correct rejection plus the frequen- 
cy of correct acceptance, whilne the overall number of in-^ 
correct decisions or "error rate* ^is the frequency of^ /false 
acceptance plus the frequency of false rejection. . The^ goal 
of decision theory i s to increase correct rate and decrease 
error rate . 

Incre3,sing the cut-off >score, e.g., ,to 75 , will simul- 
taneously decrease^ the false acceptancel error and increase 
the. false rejection error. And, decreasing the cut-off score, 
e.g., to 65 , will simultaneously decrease the, false -^ejection 
error and increase the false acceptance error. Increasing 
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F±g. 1. Frequency dis-^ibutions o¥^ test scores for masters 
and non-rjasters with cut-oXf score of 70 indicated. 
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and decreasing the cut-off point i^ illustrated 411 Figures t 
2 and' 3$ respectively. Tlierefore, simply increasing or ' - 



Insert Figures 2 & 3 



^ecreasin^ the cut-off point, will decrease one type of errpr 
at the ^ expense of the other. One would then have to decide 
whether increasing false acceptance or fa^se rejection ^jb' 
more serious. . ' \ . 

Figures 1 tlirough 3 can^be represented in decision 
Bjatrices. The decision matrices fori the various cut-off 
points are presented in Table 1," It should be noted that, 



Insert Table 1 



in the examples given, the numbet of masters and pon- 
masters vrere equal> i.e., P(M) = P(NM) = .50 with regard 
to- the total number of students uged. The vertical cplumn 
totials reflect tjie equality or inequality of group ^ize. 
The c^ll entriei^ canf beat^b# described as joint probabil- 
ities, as presented for cut-off score of. 65: 

' • ^ .50 X .90 = , ..45 ' 

, • , ' ' .50 X .10 = .05 ' 

- PCNM^gO My) = 'p(lJM^g).x P(^j l.HMjg) 

.50 x ' .40 = 420 



PdJM^gH iJMj) = P(Nljljg)- X P(NMj| NM^g) 

•<3 . . .50 x .60 =; .30 ^ 

The probability ^^of^correct judgment = .75, i..e., P(M^g'n%j.) 
+ P(NMjgn NMj) ;' and, the probability of incorrect judgment = 
.25, i.e., P(M^g*niJM^) + PdJMjgR Mj)w It is then possible to 
compare correct and terror rates per each cut-off point. 

As indicated above, ^ one type of -error may be judged as 
more serioit^, i.e., the experimenter .may be able -^^subjec- 
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Fig. 2 . \ Prequenfsy distributions of test scores for masters 
and masters w±>th cut-off score increased to 75. ^ 
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" Pig. 3. Frequency .distributions of test scores for masters 
and non-masters with cut-off score decreased to -65. 




TABLE I . 

iDECISION MATRICES FOR THE VARIOUS 
CUT-OPP POINTS 
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. tively or objectively quantify the relative weight of the 
two types of error. For example, while no error (0) is ' 
enta:j.le(} in correct rejection or correct acbeptancey the 
relative seriota^ness of false acceptance may be judged 
twice, that of false rejection* Therefore, fals^e ^accept- 
ance woiald be assigned a^ illative weight of 2 and false 
rejection woiald be 'assigned a relative weight of 1, That ^ 
is, as based on availalSle data from students and fabulty, 
it may be judged twice as^ serious to misclassify ^ non- 
liiaster than a master. While the"^ mlsclassif ied master would 
have to repeat the material, the mi'sclassl^^ied non-master, 
erroneously judged as competent, may not be exposed to the 
materials In the educational field,, erroneously labeling 
incompetent 'student-teachei:^ as competent could' have a 
significant effect on pupil pertf ormance, while erroneously 
labeling competent sttide^nlj-^teachers as incompetent would 
result in retraining an^ rete^ting. ^Therefore^ from a 
progpram* s short- and long-term efficiency point of view^ 
the number of misclassif ications or decisioii errors should 
be minimized. Given such serious consequences of decision 
errors, particularly false acceptance, a cut-off point 
coitLd be increased high enough such that the only decision 
errors made concern masters, - i#e. , all non-masters are, 
carreptly rejected. . , 

Should there be a judged difference in seriousness 
between false acceptance and false rejection, selection of 
a specific cut-off paint relies on reduction of the more 
serious error. As indicated, shifting cut-off points reduces 
one errpr at the expense of the other. Therefore, thje / 
direction of shift should be in the direction of the more 
serious error. For example, if false acceptance is judged 
twice as serious as false rejection, -fchen the judged serious- 
ness of the error would =^ 2( false acceptance) + 1 (false 
rejection). In other words, the probability of an error 
should take into account its judged seriousness. Table II 
presents the correct and error (incorrect) decision rates 
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with false acceptance ,being judged equally and twice aa 
serious as falpe rejection. A. cut-off score of 75 iould 
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be indicated, particularly for the 2(PA) + 1(FR) situation. 

In addition to 'jx^int probabilities, conditional or 
dependent probabilities can .T?e obtained per group or, per test 
score% * Joint probabilities ;are read and,' e.g., PCBHrpgH Mj) 

^ stages the probability that a student is both a master in 
the true state and also judged a master. Conditional prob- 
abilities, however, are read 'given thatj^» e.g., ^(^j \ ^T3^ 
states th« probability that a student is Judged a ^master 

' given that he is a'^naster in the true state.^^It sh^^d be 
noted that the refei^^nce grpup is different for the two types 
of •probabilities. The former is based on the total nuifiber 
of stijident^ used, both masters and non-masters, while the 
latter is bas^d solely on the subgroup of masters. 

With the distributions presented: in Pigufe 1, it is 
possible to obtain a series of conditional probabilities. 
For example^^ Table III ^lists, t^e conditllonal probabilities 
of non-masters and masters relevant at each given ^test 
score. One would expect a decrease -in the probability of- 



Insert Table III ' A 



iion-master with increasing test scores; and, likewise, one 
wouXd*^ expect an increase in the probability qf master with 
increasing test scores. This is illustrated in Table lY. 
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It should be noted that both Table IJl & ;EV deal with the 
probability of non-master and master at each given test 
score. Simply, this is the frequency of non-masters at 
score X and the frequency of masters at .score X each divided 
^'by the total number of students at score X. 



. • TAB.LE II 

CORBEd^ & -ERROR (INCORRECT) DECIS I qjT RATES AT 
VARIOUS CUT-OFF POINTS AND DIFI^RENT 
DJEGREES OF ERROR SERIOUSNESS , 
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TABLE. Ill 

CONDITIOHAIi, PROBABILITIES OF NONj^MASTERg 
AND MASTERS GIVEN' A :TEST SCORE 



P(NM 
P(NM 
P(NM 
P(NM 

P(NM 
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P(NM 
P(NM 



30) 
40) 
50) 
60)' 
70)* 
80) 
90) 
100) V 



0) 
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p(m1 30) 
P(M I 40) 
P(M I 50) 
P(M j 60) 
P(M I . 70)*« 
P(M I 80) • f 
P(M I 90) 
P|fM-|lOO) \^ 
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♦probability of a non-master given teet score of 70. 

**probabilityi of a master given test score of 70. 
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TABLE 17 y . 

EXAMPLE 'OP CORDITIOML PROBABILITIES 
AT EACH TEST SCORE . ' . 



PCNMI 30) 

P(HM| 35) 
P(NM| 40) 

P(NM| >5) 
P(HM| 50) 
P(HM| 55) 
T(liM| 60) 
P(NM| 65) 
P(»M| 70) 
P(HM| 75)**= 
P(NM| 80) 
P(liM| 85) = 
P(WMl '90) 
P(KM| 95) = 
P( KM I 100) 



m 

]JA 

100% 
100% 
100% 

95% 
■ 80% 

■;>o% 

^ 20% 
, • 5% 
' 0%° 
0% 

■ 0%' 



= HA 



r 



P(MJ ^30) = 
P(M| 35) .= 
P(M| 40) = 
P(M| 45) = 
P(M| 50) . = 
P.(M| 55} = 
P(M| 60) = 
P(M|;65) = 
P(M| 70)* = 
P(mI 75j**- 
P(M| 80) ' = 
P(M| 85) = 
P(Mi 90) = 

P(M| 95) = 
P(M|100) 



KA . 

NA' 
0% 
0% 

O^v 

20% 



80% 
^95% 
100% 
10^ 



= NA' 



^of the students scoring. 70, 50% were non-masters and 50% 
were masters • ' 

**of the students scoring 75, 20% were non-masters and SOJ^ 
wer« masters - " , ' 



4- 



\ 



ERIC 



■V- * 



,also J)ossible to obtain conditional probabilities 
of non-masters and masters for a given score and above • This 
is illustrated in Table V. Calctilation, in this case, wpuld 
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consist of the frequency of non-masters at or a^^ove score 
^d the frequency of masters at. or above score^X eadh 
divided by the total number of ^students at or above score 
X. I 

Furthermore, the conditional probabilities of non- 

masters and masters obtaining scores below X are obtain- 

• . - " ■ • " * ^ 0 

able;' In this case, the frequency of no|i-masters scoring 

less than X and the frequency of masters scoring less than 
X are each divided by the total number of students scoring 
less than.?. For example, P(im | < 70) E3(id P(^l< 70) refer to 
the probability of non-masters and ma sifgrsVl^ 
for sttudents Woring less than 70. Add! t lonelily, the con- 
ditional probabilities of sco2*lng at or a^^oVe and belb^ 
score-^ can be obtained for non-ma"ster and master groups. 
That is, for each group qf non-masters and masters, the 
frequency of scoring at or above X and the frequency of 
scoring below X are each divided by the number of students 
per group. Pot example, P(> 704 HM) and P(< 70\ UM) refer 
tc|j)the probability of s coining at or above and below 70, 
respectively, given that the student^ is a non-master. And, 
P(> 70 I M) and P(< 70 1 M) .refer to the probaljility of 
scoring at or above^and below 70, respectively, given that* 
the student is a master. The researcher, therefore^ has a 
^j^yariety of information available to Assist in the establish- 
ment of cut-off points. 

It is possible to reduce the error rate by imposing a 
&?^y area bf non-decision regarding the overlap of the tie^ 
frequency distributions; however^, this is not practical 
in an educational setting. It is also possible to reduce 
both types of error simultaneously by increasing the distance 



- EXAMPLE OP COirUlTIONAL PROBABILITIES 
AT EACH TEST . SCORE ANI> HIS^R 

PJCmU 30) = 50% P(M|^ 30) = 50% 

P(I3MI> 35) ■ = 50% P(M|S 35) = 50% 

P(M|> 40) = 50%", P(M|> 40) = 50% 



PdSMU 45) , = 50%^ ^ P(M|^ 45) i= 5091 

P(liM|> 50) =-^5% " V(M|^ 50) = 55% 

P(]SIMl> 55) =W ' P(Ml^ 55) = 6094 

P(Ml^ 60) = 35% 60) ^= 65% 

P(MM|^ 65) = 30% P(Ml^ 65) = 70% 

P(HMU 70)* = 25% * ^ P(mU 70)'» = 75% 

P(MM|^ 75)**= 5%. ' P(Ml^-75)*»= 85% 

PCHMI^ 80) '= 0% ■ , P(M|^ 80) = 100% 

■P(HM|^ 85) = '0% p(mI^ B5) =100% 

PdJMl^ 90) = 0% • ' / P(M|.^"90) ^'= i00% 

^(MM|> 95) = 0% j P(m|^ 95) = 100^ 

P(NM|>100) = 09^ P(mI^IOO) = NA 

♦of the students scoring 70 or higher, 25% were non- 
masters and 75% were masters ' 

♦♦of the students scoring 75 or higher, 5% ^ere non- 
masters and 95% were' masters 



between the two frequency distributions and/or reducing 
the variability within- each frequency distributions; 
however, these topics^ are beyond the present paperj 
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